Computer-aided detection of breast cancer: has promise outstripped performance?

The article by Gur et al. (1) in this issue of the Journal makes an important contribution to the literature on mammography screening in that it reports no difference in breast cancer detection and recall rates between mammograms read with computeraided detection and those interpreted by a single radiologist without computer-aided detection. Gur et al. present the recall rates and cancer detection rates for 56 432 screening mammography examinations interpreted before the introduction of computer-aided detection and 59 139 screening mammography examinations interpreted after the introduction of computeraided detection in their academic radiology practice. No statistically significant differences were observed in those rates. This finding is both remarkable and disappointing, given that computer-aided detection technology has been approved for use by the U.S. Food and Drug Administration (FDA) and is already widely used in clinical practice. Its widespread promotion was almost a promissory note to the public that it would outperform unaided radiologists. How could the study conducted by Gur et al. result in findings so different from initial evaluations of this technology considered in the FDA approval process, which reported increased breast cancer detection rates with computer-aided detection? As Gur et al. noted, earlier studies of computer-aided detection relied on retrospective interpretations of selected breast cancer cases evaluated in a controlled laboratory testing situation rather than on prospective detection of breast cancer in a setting more similar to actual practice, as they have done. In addition, early studies on computer-aided detection were typically performed using a sample of mammography examinations that had a higher prevalence of breast cancer than that found in ordinary screening practices. Such differences in study design can result in vastly different findings. Indeed, there is reason for concern that a technology intended for a particular purpose (screening) was approved on the basis of results from studies whose designs did not reflect how that technology was intended for use in actual practice. The evaluation of any new medical technology should be an ongoing process. The history of medical science has taught us that we should put our assumptions to the test in the real world. Unfortunately, evaluation of new technologies for cancer screening purposes is especially challenging for two reasons. First, a large number of individuals must often be screened before even one cancer is detected. Consequently, early studies (including those evaluating computer-aided detection) often assess new technology using laboratory or ‘testing’ situations that are augmented with cancer cases. Second, the most important clinical outcome in studies evaluating cancer screening technologies is overall survival, but this outcome requires many years of follow-up after the screening test is performed to observe possible effects on mortality. Because these data are not yet available for computer-aided detection, this technology has been evaluated using the more intermediary outcomes of cancer detection rates and cancer stage at time of diagnosis. It is especially intriguing to consider how quickly computeraided detection has been adopted in the United States. This rapid uptake by the medical community may be due, in part, to financial considerations. For example, the following appeared on the Web site of the manufacturer of one computer-aided detection system (2): Computer aided detection (CAD) can contribute to the earlier detection, by an average of over one year, of almost one out of four breast cancers. With the advent of federal and third party reimbursements for computer aided detection of breast cancer procedures, offering CAD can make both clinical and economic sense for the women’s health center. Using the example below, for a center with a caseload of fifty patients per day, every dollar invested in a CAD solution could return $7.54 in new revenues. On this basis, the payback period for a CAD solution could be under nine months. Has the new revenue promised by manufacturers of computer-aided detection fostered the widespread use of such technology before its effectiveness and safety have been adequately tested? This is an important issue facing all new technologies spawned in our market-driven economy. The safeguard, of course, is evidence-based medicine, which purports to guide clinicians to clinically sound and cost-effective decisions based on valid and relevant research. We more closely approach this utopian goal with the data provided by Gur et al. (1) on computer-aided detection in actual screening practice. The standard to which we hold new, expensive technologies such as computer-aided detection should be high. However, questions remain about the efficacy of computer-aided detection, these new data notwithstanding. Several limitations of the study by Gur et al. (1) merit discussion. First, Gur et al. could not adjust their analysis for characteristics of the women screened in their study, which may have affected their findings. They reported a stable recall rate after the introduction of computer-aided detection, but noted that more women in the later years of the study period returned for subsequent screening examinations. It is generally the case that the recall rate for subsequent examinations (prevalence screens)